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The  flash  flood  guidance  system  (FFGS)  is  an  operational  system  which  assists  forecasters  to  issue  flash 
flood  warnings  and  watches  over  the  conterminous  United  States.  Currently,  it  is  a  deterministic  system: 
given  a  specified  precipitation  accumulation  over  a  basin  and  over  a  time  period,  issuing  of  flash  flood 
watches  and  warnings  is  considered  depending  on  the  exceedance  of  a  certain  threshold  value  (flash 
flood  guidance  -  FFG).  For  given  channel  characteristics  and  initial  soil  moisture  conditions,  FFG  values 
are  computed  with  the  use  of  a  hydrologic  model. 

The  purpose  of  this  study  is  to  consider  the  effects  of  radar-rainfall  and  flash  flood  guidance  uncertain¬ 
ties  on  the  FFGS.  The  errors  in  the  FFG  are  accounted  for  by  quantifying  the  uncertainties  due  to  the  esti¬ 
mation  of  the  hydraulic  and  terrain  characteristics,  and  the  hydrologic  model  parameters  and  initial  state. 
To  account  for  the  uncertainties  in  radar-rainfall,  the  authors  use  an  empirically-based  radar-rainfall 
error  model.  This  model  requires  calibration  for  each  application  region  and  it  accounts  for  range  effects, 
synoptic  conditions,  space-time  resolutions,  and  the  spatial  and  temporal  dependences  of  the  errors. 
Thus,  the  combined  effects  of  uncertainty  in  both  radar-rainfall  and  FFG  can  be  assessed.  The  results 
are  exemplified  through  two  cases  in  a  small  basin  in  the  Illinois  River  Basin.  The  potential  of  transform¬ 
ing  FFGS  into  a  probabilistic  system  is  also  discussed. 

©  2010  Elsevier  B.V.  All  rights  reserved. 


Introduction 

According  to  the  American  Meteorological  Society  (AMS;  AMS, 
2000),  flash  floods  are  characterized  by  small  spatial  and  temporal 
scales,  generally  less  than  1000  km2  and  less  than  6  h.  Despite  the 
small  space-time  scales,  flash  floods  “continue  to  be  one  of  nature’s 
worst  killers"  (AMS,  2000),  taking  a  high  human  and  economic  toll 
(e.g.,  Senesi  et  al„  1996;  Baeck  and  Smith,  1998;  Chang,  1998;  Pet¬ 
ersen  et  al„  1999;  Pontrelli  et  al„  1999;  Ogden  et  al„  2000;  Pielke 
and  Downton,  2000;  Smith  et  al„  2000,  2001;  Weaver  et  al„  2000; 
Scofield  and  Kuligowski,  2003;  O’Connor  and  Costa,  2004;  Delrieu 
et  al.,  2005;  Downton  et  al.,  2005;  NRC,  2005;  Chancibault  et  al., 
2006;  Borga  et  al.,  2007;  Choi  et  al„  2007;  Carpenter  et  al.,  2007; 
Ashley  and  Ashley,  2008;  Jessup  and  DeGaetano,  2008). 

The  study  of  forecast  uncertainty  for  flash  floods  has  gained 
momentum  in  the  last  few  years  (e.g.,  Krzysztofowicz,  2001; 
Ferraris  et  al„  2002;  Siccardi  et  al„  2005;  Taramasso  et  al.,  2005; 
Georgakakos,  2006;  Ntelekos  et  al.,  2006;  Collier,  2007;  Reggiani 
and  Weerts,  2008;  Norbiato  et  al.,  2008,  2009;  Morin  et  al„ 

*  Corresponding  author. 

E-mail  address:  gvillari@princeton.edu  (G.  Villarini). 

0022-1694/$  -  see  front  matter  ©  2010  Elsevier  B.V.  All  rights  reserved, 
doi:  1 0.1 01 6/j.jhydrol.2010.02.01 4 


2009),  since  a  deterministic  system  provides  the  user  or  the  deci¬ 
sion  maker  with  an  “illusion  of  certainty”  (Krzysztofowicz,  2001). 
As  acknowledged  in  the  policy  statement  by  AMS  (2000),  impor¬ 
tant  research  challenges  include  “quantifying  forecast  uncertainty 
by  providing  probabilistic  forecast  guidance"  (AMS,  2000).  How¬ 
ever,  the  research  in  this  field  is  still  in  its  early  stages. 

This  study  presents  a  probabilistic  framework  that  is  applicable 
to  the  real-time  flash-flood  forecasting  problem.  In  particular,  we 
focus  on  the  flash  flood  guidance  system  (FFGS)  used  by  the  Na¬ 
tional  Weather  Service  (NWS)  to  assist  with  decisions  regarding 
flash  flood  warnings  and  watches  over  the  conterminous  United 
States.  In  the  current  deterministic  version  of  the  FFGS  system, 
the  amount  of  rainfall  of  a  given  duration  necessary  to  cause  flood¬ 
ing  conditions  at  the  catchment  outlet  is  compared  with  the  corre¬ 
sponding  radar-based  rainfall  estimates  produced  by  the  Next 
Generation  Radar  (NEXRAD)  stations  (e.g.,  Crum  and  Alberty, 
1993;  Klazura  and  Imy,  1993;  Fulton  et  al„  1998).  In  brief,  the  sys¬ 
tem  consists  of  two  components:  the  main  component  is  related  to 
the  estimation  of  the  flooding  conditions  using  hydrologic  and 
hydraulic  inputs  for  given  initial  soil  moisture  conditions.  The 
second  component  is  related  to  the  operational  estimation  of 
radar-rainfall  (see  Ntelekos  et  al.  (2006)  for  a  more  detailed 
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description  of  the  FFGS).  The  FFGS  helps  to  decide  whether  a  flash 
flood  watch  or  warning  should  be  issued  by  comparing  the  basin- 
average  radar-rainfall  estimate  with  respect  to  the  flash  flood  guid¬ 
ance  ( FFG ),  which  represents  the  rainfall  necessary  to  cause  flash 
flooding  for  given  soil  moisture  conditions.  As  mentioned  in  Ntele- 
kos  et  al.  (2006),  no  action  is  taken  if  FFG  is  much  larger  than  the 
observed  or  forecasted  rainfall,  while  issuing  a  flash  flood  warning 
is  considered  if  FFG  is  smaller  than  the  rainfall. 

Each  component  is  affected  by  various  sources  of  uncertainty  that 
could/should  be  accounted  for.  Using  different  representations  of  the 
operational  hydrologic  model,  Ntelekos  etal.  (2006)and  Georgakakos 
(2006 )  characterized  the  combined  uncertainties  associated  with  the 
core  part  of  the  system,  namely  the  calculation  of  the  FFG  values. 
These  uncertainties  included  the  combined  effect  of  hydrologic, 
hydraulic  and  initial  soil  moisture  conditions.  Ntelekos  et  al.  (2006) 
concluded  that  they  can  be  effectively  described  by  a  lognormal  dis¬ 
tribution,  for  the  cases  examined.  The  authors  also  noted,  “the  prob¬ 
lem  of  uncertainty  quantification  of  the  entire  FFGS  requires 
consideration  of  several  additional  components”.  In  particular,  they 
assumed  that  the  radar-rainfall  estimates  were  error-free,  even 
though  it  is  well-acknowledged  that  these  estimates  are  affected  by 
several  sources  of  uncertainty  (e.g.,  Wilson  and  Brandes,  1979;  Kra- 
jewskiand  Smith, 2002;  Germann  etal., 2006;  Ciach  etal.,  2007;  con¬ 
sult  Villarini  and  Krajewski,  (2010a)  for  a  recent  literature  review  of 
this  topic).  To  address  this  issue,  Ciach  et  al.  (2007)  proposed  an 
empirically-based  error  model  able  to  account  for  uncertainties  asso¬ 
ciated  with  radar-based  estimates  of  rainfall.  Based  on  this  model,  Vil¬ 
larini  et  al.  (2009a)  presented  a  generator  of  probable  true  rainfall 
fields  conditioned  on  radar-rainfall  maps.  These  results,  in  conjunc¬ 
tion  with  the  findings  by  Ntelekos  et  al.  (2006),  allow  the  transforma¬ 
tion  of  the  FFGS  from  deterministic  to  probabilistic,  which  accounts 
for  the  uncertainties  due  to  radar-rainfall  estimation,  as  well  as  the 
hydrologic  and  hydraulic  components.  This  probabilistic  framework 
would  enable  decision  makers  to  make  more  informed  decisions. 

It  is  worth  clarifying  that  in  this  study  we  are  focusing  on  the 
real-time  flash  flood  problem,  in  which  the  observed  radar-rainfall 
is  compared  against  FFG  (see  also  Martina  et  al.  (2006),  Ntelekos 
et  al.  (2006),  Norbiato  et  al.  (2008),  and  Norbiato  et  al.  (2009)).  In 
forecasting  of  flash  floods,  quantitative  precipitation  estimates 
could  be  obtained  by  means  of  Numerical  Weather  Predictions 
models,  short-term  forecasting  (or  nowcasting)  techniques,  or 
using  radar-rainfall  as  a  persistence  surrogate  for  future  rainfall 
(e.g.,  Collier,  2007).  The  uncertainties  in  the  selected  forecasting 
technique  would  add  to  the  uncertainties  in  the  radar-rainfall  esti¬ 
mates,  making  the  flash-flood  forecasting  problem  even  more 
uncertain.  Moreover,  even  though  in  operational  conditions  ra¬ 
dar-rainfall  estimates  are  used  in  the  calculation  of  the  initial  soil 
moisture  conditions,  in  this  study,  we  do  not  investigate  how  ra¬ 
dar-rainfall  errors  affect  the  soil  moisture  conditions. 

The  paper  is  organized  in  the  following  way.  In  the  next  section, 
we  describe  the  models  of  the  uncertainties  in  radar-rainfall  esti¬ 
mates  and  FFG.  In  ‘Data’,  we  present  two  illustrative  cases.  In  ‘Re¬ 
sults’  these  cases  are  analyzed  under  four  different  scenarios:  (i) 
deterministic  in  FFG  and  radar-rainfall  (current);  (ii)  probabilistic 
in  FFG  and  deterministic  in  radar-rainfall;  (iii)  deterministic  in 
FFG  and  probabilistic  in  radar-rainfall;  and  (iv)  probabilistic  in  both 
FFG  and  radar-rainfall.  ‘Discussion  and  conclusions’  summarizes 
the  main  points  and  concludes  the  paper. 


Modeling  of  uncertainties 

In  this  section,  we  briefly  describe  the  error  models  used  to  ac¬ 
count  for  the  uncertainties  in  both  the  FFG  and  the  radar-rainfall 
estimates.  For  more  details,  the  reader  is  pointed  to  Ntelekos 
et  al.  (2006),  Ciach  et  al.  (2007),  and  Villarini  et  al.  (2009a). 


Uncertainties  in  flash  flood  guidance 

The  NWS  established  the  FFGS  as  a  tool  to  assist  forecasters 
with  decisions  pertaining  to  flash  flood  watches  and  warnings  over 
the  conterminous  United  States.  Its  backbone  is  represented  by  the 
threshold-runoff  ( Thresh-R )  component,  which  corresponds  to  the 
amount  of  effective  rainfall  that  is  uniformly  distributed  over  a  cer¬ 
tain  basin  and  is  capable  of  causing  flooding  conditions.  Carpenter 
et  al.  (1999)  computed  it  by  equating  the  peak  runoff  rate  obtained 
using  the  Geomorphologic  Unit  Hydrograph  theory  (e.g.,  Rodri- 
guez-Iturbe  and  Valdes,  1979;  Rodriguez-Iturbe  et  al.,  1982a, b)  to 
the  bankfull  flow  based  on  Manning’s  steady  uniform  resistance 
formula.  For  a  given  rainfall  duration  (hourly,  three-hourly,  and 
six-hourly)  Thresh-R  is  a  one-time  task  and  has  been  computed  off¬ 
line  for  all  of  the  basins  for  which  FFGS  is  effective.  By  using  the 
Sacramento  Soil  Moisture  Accounting  Model  (SAC-SMA;  Burnash, 
1995)  and  initial  soil  moisture  conditions  provided  by  the  River 
Forecasting  Centers  (RFCs)  every  6  h,  rainfall-runoff  curves  are 
generated  in  a  “what  if’  scenario.  The  (future)  rainfall  that  causes 
runoff  equal  to  Thresh-R  under  specific  soil  moisture  conditions  is 
the  FFG  value  of  the  basin,  for  a  certain  rainfall  duration  (e.g., 
Georgakakos,  2006;  Ntelekos  et  al.,  2006). 

Ntelekos  et  al.  (2006)  focused  on  the  characterization  of  the 
uncertainties  associated  with  FFG.  First,  they  showed  that  Thresh- 
R  is  a  nonlinear  function  of  two  classes  of  data:  those  that  can  be 
derived  by  Geographical  Information  Systems  (GIS),  including 
drainage  area,  main  stream  length,  and  Horton’s  length  ratio,  and 
those  derived  from  regional  relationships  (Carpenter  et  al.,  1999), 
including  the  channel  top  width  and  the  hydraulic  depth  at  the 
outlet,  and  the  local  channel  slope.  To  study  the  uncertainty  asso¬ 
ciated  with  the  Thresh-R  calculation,  uncertainty  information 
should  be  obtained  for  those  variables.  For  the  first  class  of  vari¬ 
ables,  there  is  very  limited  information  on  the  shape  of  the  error 
distribution.  Consequently,  Ntelekos  et  al.  (2006)  assumed  general¬ 
ized  beta  error  distributions  and  decided  the  uncertainty  bounds 
based  on  data  from  previous  studies  (Carpenter  and  Georgakakos, 
1993).  For  the  class  of  variables  obtained  from  regional  relation¬ 
ships,  uncertainty  information  existed  for  the  case  of  Oklahoma, 
and  the  error  distribution  of  these  variables  (lognormal)  was  di¬ 
rectly  derived. 

The  total  uncertainty  in  the  Thresh-R  calculation  was  propa¬ 
gated  in  the  FFGS  with  the  use  of  a  simplified  version  of  the  SAC- 
SMA  model.  Additional  uncertainty  in  the  model  parameters  and 
initial  state  was  assumed  to  account  for  total  uncertainty  of  the 
simplified  FFGS.  The  authors  showed  that  the  total  uncertainty  in 
the  FFG  calculation  could  be  well  described  by  a  lognormal  distri¬ 
bution  under  two  initial  soil  moisture  conditions  (dry  and  wet). 

Uncertainties  in  radar-rainfall  estimates 

Ciach  et  al.  (2007)  proposed  an  empirically-based  radar-rainfall 
error  model,  in  which  the  relation  between  true  ground  areal  rain¬ 
fall  Ra  and  radar-rainfall  Rr  was  described  by  the  product  of  a  sys¬ 
tematic  distortion  function  and  random  component: 

Ra  =  h(Rr)s(Rr)  (1) 

where  the  systematic  distortion  function  h(Rr )  accounts  for  system¬ 
atic  errors  (bias  function  of  the  radar-rainfall  estimates),  while  the 
random  component  e(Rr)  describes  the  remaining  random  uncer¬ 
tainties.  Both  components  of  the  model  are  a  function  of  radar-rain- 
fall  values,  and  the  true  rainfall  was  approximated  by  rain  gauge 
measurements. 

As  a  preliminary  step,  the  overall  bias  (defined  as  the  ratio  be¬ 
tween  rain  gauge  and  radar  sample  means)  was  computed  and  re¬ 
moved  (it  depends  on  the  distance  from  the  radar  and  season,  used 
as  proxy  for  rainfall  regime).  Using  a  non-parametric  approach,  the 
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systematic  component  was  computed  as  an  expectation  function 
conditioned  on  the  radar-rainfall  estimates  (see  also  Villarini 
et  al.  (2008)).  Once  h  has  been  computed,  the  calculation  of  the 
random  component  is  straightforward.  To  obtain  a  more  concise 
description  of  both  the  systematic  and  random  components,  Ciach 
et  al.  (2007)  approximated  the  radar-rainfall  uncertainty  model 
with  parametric  models.  They  found  that  the  systematic  distortion 
function  can  be  parameterized  by  a  power  law  function.  They  also 
found  that  the  random  component  could  be  described  by  a  Gauss¬ 
ian  distribution  with  mean  equal  to  1 ,  a  standard  deviation  that  is  a 
monotonically-decreasing  (hyperbolic)  function  of  radar-rainfall 
values,  and  with  spatio-temporal  dependencies  that  can  be  well 
described  by  a  two-parameter  exponential  correlation  function 
(Villarini  et  al.,  2009a).  Therefore,  all  of  this  error  model’s  compo¬ 
nents  have  simple  parametric  representations.  To  account  for 
range  effects,  Ciach  et  al.  (2007)  divided  the  radar  umbrella  into 
five  zones,  while  they  accounted  for  synoptic  conditions  by  strati¬ 
fying  the  sample  into  three  seasons:  cold  (January,  February, 
March,  November,  and  December),  warm  (April,  May,  and  Octo¬ 
ber),  and  hot  (June,  July,  August,  and  September).  The  results  pre¬ 
sented  by  Ciach  et  al.  (2007)  are  based  on  a  large  sample 
(6  years)  of  radar  data  from  the  NEXRAD  site  in  Oklahoma  City 
(KTLX),  hourly  and  larger  time  scales,  averaged  over  the  Hydrologic 
Rainfall  Analysis  Project  (HRAP)  grids  (approximately  4  km  by 
4  km  pixels:  Reed  and  Maidment,  1999),  and  NEXRAD  reflectiv¬ 
ity-rainfall  ( Z-R )  relation  (Fulton  et  al.,  1998).  The  radar  estimates 
are  complemented  with  rainfall  measurements  by  the  Oklahoma 
Micronet  (e.g.,  Allen  and  Naney,  1991)  and  Mesonet  (e.g.,  Brock 
et  al.,  1995)  rain  gauge  networks. 

Building  on  the  results  by  Ciach  et  al.  (2007),  Villarini  et  al. 
(2009a)  developed  a  generator  of  ensembles  of  probable  true  rain¬ 
fall  fields  conditioned  on  a  given  radar-rainfall  map.  Taking  advan¬ 
tage  of  the  Gaussian  nature  of  the  random  component,  it  is  also 
possible  to  create  probabilistic  maps,  which  provide  information 
on  the  probability  that  the  true  rainfall  is  larger  than  a  certain 
threshold  of  interest  Rthres.  Since  we  are  interested  in  the  probabil¬ 
ity  of  exceedance  of  the  flash  flood  guidance  by  the  true  rainfall 
over  a  certain  basin,  we  use  the  following  analytical  expression 
from  Villarini  et  al.  (2009a): 


p(±  £**>*■«) 


=  1  -  cD(x)  =  l-0  " 

^  JTLA  a\i  +  EIUEJU  J 

(2) 

where  i  and  j  are  indexes  referring  to  certain  pixels  among  the  n 
number  of  pixels  within  the  area  of  interest  (flash  flood  prone 
catchment  in  this  case),  Rthres  is  the  flash  flood  guidance  value  for 
a  certain  basin  and  soil  moisture  conditions,  er£  and  ps  are,  respec¬ 
tively,  the  standard  deviation  and  the  spatial  correlation  of  the  ran¬ 
dom  error  component,  and  <!>(■)  is  the  standard  normal  cumulative 
distribution  function. 

This  error  model  has  already  been  applied  towards  improving 
statistical  validation  of  satellite  precipitation  estimates  (Villarini 
et  al.,  2009b)  and  to  investigate  the  impact  of  radar-rainfall  uncer¬ 
tainties  on  the  scaling  properties  of  rainfall  (Villarini  and  Krajew- 
ski,  2009b). 


Combining  radar-rainfall  and  FFG  uncertainties 


help  decide  whether  (or  not)  to  issue  a  flash  flood  watch  or  warn¬ 
ing.  When  we  consider  only  the  uncertainties  in  FFG  and  a  deter¬ 
ministic  radar-rainfall  value,  we  can  compute  the  probability  that 
basin-average  rainfall  is  larger  than  the  FFG  value.  The  same  can 
be  done  if  we  account  for  the  uncertainties  in  radar-rainfall  and 
consider  the  FFG  value  as  deterministic.  However,  when  account¬ 
ing  for  uncertainties  in  both  of  the  components  of  the  FFGS,  we 
compare  quantiles  qFFc  from  the  FFG  distribution  against  quantiles 
qRR  from  the  radar-rainfall  distribution.  For  any  pair  (qrrc,  Qrr),  we 
can  check  whether  the  FFG  value  is  larger  than  the  radar-rainfall 
value.  If  decisions  are  based  only  on  the  comparison  between  FFG 
and  radar-rainfall  values,  we  can  create  warning-no  warning  plots 
depending  on  whether  the  FFG  value  is  larger  than  the  radar-rain- 
fall  one. 

However,  users  and  forecasters  will  need  to  make  deterministic 
decisions  regarding  whether  to  issue  a  flash  flood  warning  or 
watch.  The  selection  of  an  FFG  and  radar-rainfall  quantile  is  dic¬ 
tated  by  both  economic  and  societal  issues.  The  selection  of  a  high 


Rr  quantile 


As  mentioned  before,  in  the  current  deterministic  FFGS  the  FFG 
value  is  compared  to  the  basin-average  radar-rainfall  estimates  to 


Fig.  1.  Plots  with  the  probability  of  missed  warnings  (upper  panel)  and  the 
probability  of  false  alarms  (bottom  panel)  for  different  FFG  and  radar-rainfall 
quantiles. 
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FFG  quantile  leads  to  a  higher  probability  of  missed  warnings  (a 
flash  flood  occurred  but  no  flash  flood  warning  or  watch  was  is¬ 
sued).  This  situation  would  lead  to  a  high  toll  in  terms  of  fatalities 
and  economic  damage.  On  the  other  hand,  a  high  threshold  in  ra¬ 
dar-rainfall  quantiles  would  result  in  an  increasing  probability  of 
false  alarms,  with  high  economical  and  societal  repercussions.  A 
possible  way  of  providing  suggestions  concerning  the  selection  of 
qFFC  and  qRR  is  by  looking  at  the  probability  of  false  alarms  (PFA) 
and  the  probability  of  missed  warnings  (PMW).  Given  the  total 
number  of  warnings,  the  former  can  be  defined  as  the  probability 
of  a  warning  if  no  event  occurred.  On  the  other  hand,  the  probabil¬ 
ity  of  missed  warnings  can  be  defined  as  ratio  between  the  number 
of  missed  warnings  and  the  total  number  of  flash  floods.  We  would 
expect  a  higher  probability  of  false  alarms  for  increasing  qRR  values, 
and  a  higher  probability  of  missed  warnings  for  increasing  qFFC. 

For  each  basin  and  combination  (qFFG,  qRR)  it  would  possible  to 
compute  these  two  probabilities  from  the  data.  In  this  study,  we 
do  not  have  this  information  and  therefore  we  have  created  plots 
with  PFA  and  PMW  for  different  quantiles  and  we  have  summa¬ 
rized  the  results  in  Fig.  1.  In  this  case,  the  variation  of  PMW  and 
PFA  as  a  function  of  qFFC  and  qRR  is  arbitrary,  even  though  it  reflects 
the  idea  that  we  would  have  more  false  alarms  for  higher  qKK  (and / 
or  lower  qFFC),  and  more  missed  warnings  for  higher  qFFC  (and/or 
lower  qm). 

Similar  to  Martina  et  al.  (2006),  we  define  the  “cost”  as  the  dam¬ 
age  perception  by  the  user  or  forecaster.  This  “cost”  does  not  di¬ 
rectly  translate  into  the  economic  damage  associated  with  a  flash 
flood,  but  includes  both  economic  as  well  as  unquantifiable  dam¬ 
age  (e.g.,  societal  perception  of  false  alarms).  It  is  the  task  of  stake¬ 
holders  and  decision  makers  to  quantify  the  "cost”  associated  with 


false  alarms  and  missed  warnings.  In  this  study,  we  assume  that 
this  piece  of  information  is  available.  Following  the  definition  of 
risk  (it  is  the  product  of  the  probability  of  occurrence  of  a  certain 
event  and  the  “cost”  associated  with  that  event),  for  any  pair  (qFFC, 
qRR)  we  can  compute  the  risk  associated  with  a  false  alarm  and 
with  a  missed  warning.  We  can  then  define  the  risk  associated  with 
a  flash  flood  ( riskFF )  as  the  sum  of  the  risk  associated  with  a  missed 
warning  and  a  false  alarm.  Following  Martina  et  al.  (2006),  we  can 
then  select  the  most  “convenient"  FFG  and  radar-rainfall  quantiles 
by  minimizing  riskFF. 


Table  1 

Scenarios  of  the  initial  soil  moisture  conditions  investigated  (in  %  of  the  capacity). 

Upper  zone  tension  water  content  (%)  70  90 

Upper  zone  free  water  content  (%)  60  85 


Table  2 

Summary  of  the  values  of  the  FFG  (mm/3  h)  and  the  basin-averaged  radar-rainfall 
estimates  (mm/3  h)  for  the  two  cases  and  the  two  soil  moisture  scenarios  considered 

Case  I  Case  II 

Dry  Wet  Dry  Wet 

Flash  flood  guidance  (mm/3  h)  80.87  57.64  80.87  57.64 

Basin-averaged  radar-rainfall  (mm/3  h)  67.73  67.73  43.70  43.70 


Fig.  2.  Map  of  the  Illinois  River  Basir 
50  km  apart. 
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As  mentioned  above,  in  this  study,  we  assume  as  known  the  values  could  be  obtained  from  the  available  record  or  by  using  a 

probabilities  of  missed  warnings  and  false  alarms  for  different  hydrologic  model.  Moreover,  we  also  assume  that  the  “cost”  asso- 

combinations  of  FFG  and  radar-rainfall  quantiles.  In  reality,  these  ciated  with  false  alarms  and  missed  warnings  was  previously 


Fig.  3.  Results  from  the  application  of  a  flash  flood  guidance  system  probabilistic  in  FFG  and  deterministic  in  Rr  for  the  two  cases  under  study  and  two  different  soil  moisture 
conditions. 
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quantified.  For  generality,  we  assume  that  the  “cost”  associated 
with  false  alarms  is  proportional  to  the  cost  associated  with  missed 
warnings.  In  this  way,  we  can  investigate  how  the  most  “conve¬ 
nient"  FFG  and  radar-rainfall  quantiles  change  as  a  function  of 
“cost”. 

Data 

In  this  study,  we  present  the  results  of  the  methodology  of  the 
previous  section  for  the  same  basin  considered  in  Ntelekos  et  al. 
(2006).  This  catchment  [US  Geological  Survey  (USGS)  basin  cata¬ 
loguing  number  2568]  is  located  about  100  km  from  the  Tulsa 
(KINX)  NEXRAD  station  (Fig.  2)  and  is  a  sub-basin  of  the  Illinois 
River  Basin,  with  a  drainage  area  of  about  116  Ion2  and  a  stream 
length  of  1.88  km.  Since  the  selected  basin  is  not  covered  by  the 
Oklahoma  City  (KTLX)  radar,  we  assume  that  the  parameters  of 
the  radar-rainfall  error  model  by  Ciach  et  al.  (2007)  are  valid  for 
KINX  radar  as  well.  The  Tulsa  radar  (an  S-band  radar  like  KTLX) 
is  located  at  about  180  km  from  the  Oklahoma  City  radar  and  orog¬ 
raphy  is  not  a  dominant  factor  in  this  entire  area.  Since  the  results 
in  Ciach  et  al.  (2007)  are  dependent  on  the  selected  radar-rainfall 
product  and  radar,  there  is  no  certainty  that  the  parameters  for 
the  KTLX  radar  would  be  the  same  for  the  KINX  radar,  even  though 
there  is  evidence  that  the  overall  model  structure  (in  terms  of  sys¬ 
tematic  and  random  components)  would  be  appropriate  (see  Villa¬ 
rini  and  Krajewski  (2009a)  for  the  application  of  this  error  model  to 
a  C-band  radar  in  southern  England).  The  transferability  of  the  re¬ 
sults  in  Ciach  et  al.  (2007)  to  other  radars,  in  particular  KINX, 


should  be  the  object  of  future  works.  As  far  as  the  soil  moisture 
conditions  are  concerned,  we  investigate  the  same  two  scenarios 
(dry  and  wet  conditions;  Table  1)  as  in  Ntelekos  et  al.  (2006).  None¬ 
theless,  it  is  worth  mentioning  that  different  studies  in  the  litera¬ 
ture  found  that  large  uncertainties  in  the  predictability  of  the 
hydrologic  response  were  observed  for  intermediate  soil  moisture 
conditions  when  small  soil  moisture  variations  can  produce  signif¬ 
icant  changes  on  runoff  (e.g.,  Zehe  and  Bloschl,  2004;  Zehe  et  al„ 
2007;  Zehe  and  Sivapalan,  2009). 

We  focus  on  the  three-hourly  scale  and  have  selected  two  dif¬ 
ferent  storms:  Case  I  which  transpired  on  October  5th,  1998,  and 
Case  II  which  occurred  on  June  21st,  2000.  The  selection  of  these 
two  cases  is  not  associated  with  the  occurrence  of  flash  flooding 
over  the  selected  catchment,  but  these  two  cases  allow  us  to  de¬ 
scribe  the  modifications  of  the  FFGS  once  uncertainties  in  both 
FFG  and  radar-rainfall  are  accounted  for.  The  radar  data  was  pro¬ 
cessed  through  the  Hydro-NEXRAD  software  system  (Krajewski, 
2007a;  Krajewski  et  al.,  2008).  It  is  a  web-based  system  that  allows 
generation  of  radar-rainfall  products  (derived  from  NEXRAD  data 
from  about  40  sites  in  the  United  States).  Several  customized  mod¬ 
ular  radar-rainfall  algorithms  are  available  in  Hydro-NEXRAD 
(Quick  Look,  High  Res,  Pseudo  NWS  PPS,  and  Custom;  Krajewski, 
2007b).  In  this  study,  we  use  the  so-called  “Pseudo-PPS,”  which 
is  a  suite  of  algorithms  that  tries  to  mimic  the  NWS  Precipitation 
Processing  System  (PPS;  Fulton  et  al„  1998)  as  closely  as  possible. 
As  shown  by  Villarini  and  Krajewski  (2010b),  the  radar-rainfall 
modeling  results  from  “Pseudo-PPS”  favorably  compared  with 
those  by  Ciach  et  al.,  2007  using  NWS  PPS.  We  generated  hourly 
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accumulation  maps  averaged  over  HRAP  grid  and  accumulated 
them  to  obtain  three-hourly  radar-rainfall  fields.  Table  2  summa¬ 
rizes  the  values  of  FFG  and  Rr,  averaged  over  the  selected  basin,  un¬ 
der  wet  and  dry  conditions,  for  the  two  selected  cases. 


Results 

In  the  current  operational  application  of  the  FFGS,  FFG  values 
are  compared  with  radar-rainfall  estimates  or  rainfall  nowcasts 
(produced  either  using  radar  data  or  independently  of  radar  data 
through  nowcast  models),  and  this  information  is  used  with  other 
local  information  to  make  decisions  concerning  whether  or  not  to 
issue  a  flash  flood  watch  or  warning  for  a  particular  basin  and  a 
particular  time  period.  For  the  purposes  of  this  paper  we  will  as¬ 
sume  that  the  forecaster  has  available  only  the  FFGS  information 
to  make  a  decision.  According  to  the  current  system,  a  flash  flood 
warning  or  watch  should  (not)  be  issued  for  Cases  I  in  wet  (dry) 
conditions,  while  for  Case  II  FFG  is  always  larger  than  the  radar- 
rainfall  value  (Table  2).  However,  under  these  conditions,  the 
uncertainties  associated  with  radar-rainfall  estimation  and  the 
hydraulic  and  hydrologic  components  of  the  system  are  neglected. 

Similar  to  Ntelekos  et  al.  (2006),  we  evaluate  a  scenario  in 
which  we  account  for  the  uncertainties  in  FFG  but  we  still  consider 
the  radar-rainfall  estimates  error-free  (probabilistic  in  FFG  and 
deterministic  in  Rr ).  As  mentioned  above,  the  uncertainties  in  FFG 
for  this  basin  can  be  described  by  a  lognormal  distribution  with 
mean  and  variance  from  Table  1 1  in  Ntelekos  et  al.  (2006).  Fig.  3 


shows  the  results  from  our  analysis.  Case  I  resulted  in  the  largest 
rainfall  over  the  basin.  This  higher  value  is  highlighted  by  a  higher 
probability  of  exceedance  of  FFG  by  the  rainfall.  Under  dry  condi¬ 
tions,  the  probability  that  a  flash  flood  would  occur  is  equal  to 
0.31  and  0.05  for  Cases  I  and  II,  respectively.  This  probability  in¬ 
creases  under  wet  conditions:  it  is  equal  to  0.64  and  0.26,  respec¬ 
tively  for  Cases  I  and  II. 

Another  possible  scenario  is  that  FFG  is  treated  as  deterministic 
(error-free),  while  we  account  for  the  uncertainties  in  radar  esti¬ 
mates  of  rainfall  (probabilistic  in  Rr).  In  Fig.  4,  we  show  the  results 
for  this  scenario  for  the  two  cases  and  the  different  initial  soil 
moisture  conditions.  Under  these  conditions,  the  probability  that 
the  true  areal  ground  rainfall  is  larger  than  or  equal  to  FFG  is  smal¬ 
ler  than  0.01  for  both  cases  under  dry  conditions,  and  it  increases 
to  0.09  and  0.01  for  Cases  1  and  II  under  wet  conditions.  These 
probabilities  are  consistently  smaller  than  those  obtained  under 
probabilistic  FFG  and  deterministic  Rr.  Ciach  et  al.  (2007)  found 
that  the  Oklahoma  City  radar  had  the  tendency  to  overestimate 
rainfall  (in  terms  of  both  unconditional  and  conditional  biases). 
This  aspect  could  explain  the  small  probabilities  of  exceedance  of 
FFG  by  the  basin-average  rainfall  (consult  also  Georgakakos 
(2005)  for  additional  results  regarding  the  impact  of  bias  in  rainfall 
estimates  on  the  FFGS). 

The  final  step  is  the  development  of  a  scenario  in  which  we  ac¬ 
count  for  both  radar-rainfall  and  FFG  uncertainties.  As  mentioned 
above,  we  no  longer  compare  a  single  deterministic  value  with  an¬ 
other  one  or  against  a  distribution,  as  in  the  previous  three  scenar¬ 
ios.  We  compare  a  certain  quantile  Qffg  from  the  distribution  of  FFG 


k  function  (arbitrary  units)  for  a  damage  ratio  of  0.5  (upper-left  panel),  1.0  (upper-right  panel),  1.5  (lower-left  panel),  and  5  (lower-right  panel).  The 
as  the  ratio  between  damage  associated  with  missed  warnings  and  the  damage  associated  with  false  alarms. 


Fig.  6.  Plots  with  the  risl 
damage  ratio  is  defined  , 
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damage  ratio.  The  damage  ratio  is  defined  as  the  ratio  between  damage  associated 
with  missed  warnings  and  the  damage  associated  with  false  alarms. 


with  one  from  the  distribution  of  the  radar  data  qR R.  In  this  way,  we 
can  generate  plots  in  which  we  evaluate  whether  qFFC  is  smaller 
(flash  flood  warning  or  watch  issued)  or  larger  (flash  flood  warning 
or  watch  not  issued)  than  qKR.  In  Fig.  5,  we  present  the  results  for 
both  of  the  cases  under  both  dry  and  wet  conditions.  As  expected, 
as  the  value  of  the  FFG  quantile  increases,  there  is  less  chance  that 
a  flash  flood  warning  needs  to  be  issued.  The  opposite  occurs  for 
increasing  Rr  quantiles. 

The  next  step  is  the  selection  of  the  FFG  and  radar-rainfall  quan¬ 
tiles.  As  mentioned  before,  we  should  have  information  about  the 
probability  of  missed  warnings  and  false  alarms.  In  this  study, 
we  have  assumed  that  it  can  be  described  as  in  Fig.  1.  Moreover, 
we  also  assume  to  have  information  about  the  “cost”  associated 
with  false  alarms  and  missed  warnings.  Given  these  two  pieces 
of  information  we  can  compute  riskFF  for  any  FFG  and  radar-rain- 
fall  quantiles.  For  simplicity,  we  have  assumed  that  the  “cost”  asso¬ 
ciated  with  false  alarms  is  a  multiple  of  the  “cost”  associated  with 
missed  warnings  (the  “damage  ratio”  represents  the  ratio  of  the 
missed  warning  cost  to  the  false  alarm  cost).  In  Fig.  6  we  have  sum¬ 
marized  the  risk  plots  for  a  damage  ratio  of  0.5  (upper-left  panel), 
1.0  (upper-right  panel),  1.5  (lower-left  panel),  and  5  (lower-right 
panel).  Given  the  values  in  Fig.  1  and  for  a  damage  ratio  of  1  (same 
“cost”  associated  with  missed  warnings  and  false  alarms),  we  have 
that  the  minimum  value  of  riskFF  corresponds  to  a  quantile  value  of 
0.58  for  radar-rainfall  and  0.52  for  the  FFG.  For  increasing  values  of 
the  damage  ratio,  the  most  “convenient”  (see  Martina  et  al.  (2006)) 
combination  of  quantiles  moves  to  lower  FFG  quantiles  and  higher 
radar-rainfall  quantiles,  reflecting  the  higher  “cost”  associated 
with  a  missed  warning  than  with  a  false  alarm  (Fig.  7).  On  the  other 
hand,  if  the  “cost”  associated  with  false  alarms  is  larger  than  the 
“cost"  associated  with  missed  warnings,  then  the  minimum  value 
of  riskFF  moves  to  lower  qRK  values  and  larger  qFFC  values  (Fig.  7). 

In  this  example,  the  selection  of  qFFC  and  qRK  is  a  function  of  the 
damage  ratio.  However,  once  the  “cost"  of  false  alarms  and  missed 
warnings  is  established,  single  values  of  these  quantiles  can  be  ob¬ 
tained.  Once  we  have  qFFG  and  qKK,  we  can  then  use  Fig.  5  to  help 
decide  about  issuing  (or  not)  a  flash  flood  warning  or  watch. 

Discussion  and  conclusions 

The  current  FFGS  used  by  the  NWS  to  help  in  issuing  flash  flood 
warning  or  watches  is  a  deterministic  system  and  can  be  roughly 
broken  down  into  two  main  components:  the  computation  of  the 


FFG  values  and  the  radar-rainfall  estimation.  However,  both  com¬ 
ponents  are  affected  by  errors  that  need  to  be  characterized  and  ac¬ 
counted  for  to  provide  users  and  decision  makers  with  the  tools  to 
make  an  informed  decision.  Ntelekos  et  al.  (2006)  showed  how  the 
uncertainties  associated  with  FFG  could  be  described  by  a  lognor¬ 
mal  distribution,  while  Ciach  et  al.  (2007)  developed  an  empiri¬ 
cally-based  radar-rainfall  error  model.  Using  the  analytical 
formulation  by  Villarini  et  al.  (2009a)  to  account  for  radar-rainfall 
uncertainties,  the  simplified  FFGS  was  examined  under  four  differ¬ 
ent  scenarios  and  for  two  different  rainfall  events:  (i)  deterministic 
in  both  Rr  and  FFG  (current  system);  (ii)  deterministic  in  Rr  and 
probabilistic  in  FFG;  (iii)  probabilistic  in  Rr  and  deterministic  in 
FFG;  and  (iv)  probabilistic  in  both  Rr  and  FFG. 

Neglecting  the  uncertainties  and  using  a  deterministic  approach 
would  not  help  in  making  an  optimal  decision  and  would  only  give 
an  “illusion  of  certainty”  as  describe  in  Krzysztofowicz  (2001 ).  On 
the  other  hand,  a  system  in  which  the  errors  are  accounted  for 
would  allow  users  to  make  more  informed  decisions.  In  the  pro¬ 
posed  modification  of  the  FFGS  from  a  deterministic  to  a  probabi¬ 
listic  system,  we  can  generate  a  plot  in  which  the  uncertainties  in 
radar-rainfall  and  FFG  are  explicitly  accounted  for.  Thus,  users 
would  have  to  decide  whether  or  not  to  issue  a  flash  flood  warning 
or  watch  based  on  the  provided  uncertainty  information  and  their 
knowledge  and  experience  of  the  system  and  the  study  area.  Nev¬ 
ertheless,  it  is  clear  that  the  decision  concerning  the  selection  of 
qFFG  and  qRK  is  critical:  low  FFG  (or  high  Rr )  quantiles  could  result 
in  a  large  number  of  false  alarms,  with  economical  and  social 
repercussions,  while  high  FFG  (or  low  Rr )  quantiles  could  result 
in  unannounced  flash  floods,  with  a  potentially  significant  human 
toll.  From  an  economic  standpoint,  the  choice  could  be  driven  by 
the  use  of  cost-loss  models,  which  could  help  optimizing  the  deci¬ 
sion  making  process  (e.g.,  Mylne,  2002).  Therefore,  the  selection  of 
the  quantiles  should  be  based  on  a  detailed  cost-benefit  analysis 
resulting  from  different  qppG-qm  scenarios  and  should  consider 
the  land  use  and  population  of  the  area.  In  this  study,  we  have  pro¬ 
vided  some  indications  on  how  to  select  the  most  “convenient” 
values  of  quantiles  for  both  FFG  and  radar-rainfall.  The  proposed 
results  are  based  on  assumptions  concerning  the  probability  of 
missed  warnings  and  the  probability  of  false  alarms.  To  evaluate 
the  performance  of  the  methods  advanced  here,  real  data  from  his¬ 
torical  flash  floods  could  be  used  as  a  “calibration  tool”  towards  the 
selection  of  the  quantile  to  use  in  the  decision  process  for  particu¬ 
lar  regions  and  lead  times.  The  development  of  databases  of  ob¬ 
served  flash  flood  events  in  the  US  is  an  important  undertaking 
for  this  purpose  (see  discussion  in  Carpenter  et  al.  (2007)). 

To  complicate  the  matter,  there  is  also  interdependency  be¬ 
tween  the  two  components,  since  initial  soil  moisture  conditions 
are  computed  using  the  radar-rainfall  estimates.  Future  studies 
should  investigate  the  impact  of  radar-rainfall  uncertainties  on 
the  initial  soil  moisture  conditions,  and  how  these  uncertainties 
propagate  through  the  FFGS.  Initial  steps  illustrating  methodolog¬ 
ical  aspects  along  this  direction  have  been  made  in  Georgakakos 
(2005)  but  significant  research  remains  to  be  done  for  the  develop¬ 
ment  of  operationally  useful  flash  flood  warning  systems. 

Admittedly,  our  results  are  based  on  only  two  cases,  assuming 
that  the  parameters  of  the  error  model  estimated  for  the  Oklahoma 
City  radar  are  valid  for  the  Tulsa  radar  as  well.  Nevertheless,  this  is 
an  important  advancement  towards  the  transition  from  a  deter¬ 
ministic  to  probabilistic  flash  flood  alert  system.  More  effort  is  re¬ 
quired  to  extend  this  formulation  to  areas  characterized  by 
different  rainfall  regimes  and/or  more  complex  orography:  the 
characterization  of  the  errors  in  FFG  and  radar-rainfall  for  different 
areas  or  radars  is  not  yet  available  and  should  be  the  object  of  fu¬ 
ture  studies.  In  particular,  to  characterize  the  uncertainties  in  FFG 
for  a  given  catchment,  we  would  need  information  derived  from 
GIS  (drainage  area,  main  stream  length,  Horton’s  length  ratio) 
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and  from  regional  relationships  (channel  top  width  and  hydraulic 
depth  at  the  outlet,  and  local  channel  slope),  as  in  Carpenter 
et  al.  (1999).  For  describing  the  uncertainties  in  radar-rainfall 
according  to  the  model  by  Ciach  et  al.  (2007),  we  would  need  a 
large  dataset  of  rain  gage  and  radar-rainfall  data  to  compute  the 
parameters  of  the  systematic  and  random  components  of  the  error 
model.  More  generally,  the  broad  development  of  validation  pro¬ 
grams  for  flash-flood  forecasting  could  serve  the  dual  roles  of  pro¬ 
viding  quantitative  assessments  of  forecast  accuracy  and  data  sets 
that  would  be  useful  for  enhancing  models  of  FFG  error  structure. 

Error  characteristics  for  urban  catchments  can  exhibit  proper¬ 
ties  that  are  especially  difficult  to  characterize.  Javier  et  al. 
(2007)  illustrated  the  role  of  urban  infrastructure  in  determining 
error  properties  of  flash-flood  forecasting  components.  Advances 
in  monitoring  land  surface  properties  (see,  for  example,  Horritt 
and  Bates,  2002;  Lane  et  al.,  2003)  for  urban  catchments  and  inte¬ 
gration  of  new  data  sources  into  representations  of  model  error 
structure  are  important  research  areas. 

Finally,  given  the  dependence  of  the  radar-rainfall  error  model 
components  on  radar-rainfall  values,  we  can  characterize  the 
uncertainties  associated  with  extreme  precipitation  events,  which 
are  generally  the  most  interesting  in  these  applications.  However, 
modeling  of  radar-rainfall  uncertainties  for  this  type  of  events 
may  require  further  efforts  and  should  be  investigated  in  separate 
studies. 
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